1. Field of the Invention
The present invention relates to an image pickup apparatus and an image pickup method, both using an image pickup region provided with an arrangement of a plurality of pixels each including a plurality of photoelectric conversion portions arranged in a depth direction.
2. Related Background Art
An image pickup apparatus such as a digital camera ordinarily has a configuration as shown in FIG. 12. In the configuration shown in this figure, a system control central processing unit (CPU) 1200 detects a change of the state of a camera operation switch (SW) 1201 (composed of a main SW and a release SW of the camera) caused by a photographer himself or herself, and starts to supply electric power to each circuit block other than the system control CPU 1200 itself.
An image of an object in a range of a photographing screen is formed onto an image pickup element 1204 through a main photographing optical system 1202 and 1203. An electric signal from the image pickup element 1204 based on the formed image is input into a correlated double sampling (CDS)/automatic gain control (AGC) circuit 1205. The CDS/AGC circuit 1205 removes reset noises included in the input electric signal in accordance with a method such as a known correlated double sampling, and performs the clamp processing of optical black (OB) pixels to reproduce a black level. After that, the electric signal is input into an analog to digital (A/D) conversion circuit 1206 to be converted into a predetermined digital signal at every pixel in order.
Here, the image pick up element 1204 generates an output of an image signal by being driven in a predetermined way using an output from a driver circuit 1207 for performing horizontal driving and vertical driving of each pixel on the basis of a signal from a timing generator 1208 which determines the driving timing of the whole.
The CDS/AGC circuit 1205, which converts an output of the image pickup element 1204 to a predetermined signal level by processing the output in an analog processing way, and the A/D conversion circuit 1206 similarly operate on the basis of the timing from the timing generator 1208.
An output from the A/D conversion circuit 1206 is input into a memory controller 1215 through a selector 1209, which selects signals on the basis of a signal from the system control CPU 1200. The memory controller 1215 transfers all of the output signals of the selector 1209 to a frame memory 1216. Consequently, all of the image data of every photographing frame is stored in the frame memory 1216 once. Hence, the image pickup apparatus performs writing operations of all of the photographed data into the frame memory 1216 in case of continuous photographing or the like.
After the completion of an photographing operation, the contents of the frame memory 1216 storing the photographed data are transferred to a camera digital signal processor (DSP) 1210 through the selector 1209 under the control of the memory controller 1215. The camera DSP 1210 generates respective color signals of red (R), green (G) and blue (B) on the basis of each pixel data of each photographed data stored in the frame memory 1216. In an operation state before photographing, the camera DSP 1210 ordinarily transfers the results of the generation of the color signals to a video memory 1211 periodically (at every frame) to display the results in the finder of the image pickup apparatus through a monitor display unit 1212.
On the other hand, when a photographer himself or herself performs a photographing operation by operating the camera operation SW 1201, each pixel data of one frame is read from the frame memory 1216, and subjected to image processing by the camera DSP 1210. And then, the processed pixel data is stored in a work memory 1213 once.
Successively, the data stored in the work memory 1213 is compressed on the basis of a predetermined compression format by a compression and expansion unit 1214, and the results of the data compression are stored in an external nonvolatile memory 1217 (ordinarily a nonvolatile memory such as a flash memory is used).
Moreover, when photographed image data is observed, the compressed data stored in the external nonvolatile memory 1217 is expanded to be ordinary data of each photographing pixel by the compression and expansion unit 1214, and the results of the expansion of the data are transferred to the video memory 1211. Thereby, the image data can be observed with the monitor display unit 1212.
As described above, an ordinary digital camera is configured to convert an output of the image pickup element 1204 into actual image data with process processing circuits almost in real time, and to output the result of the conversion of the output to the nonvolatile memory 1217 or the monitor display unit 1212.
In case that the image pickup element 1204 in the digital camera as described above is an multi-layer photodiode type color image sensor as shown in FIGS. 13A and 13B (see U.S. Pat. No. 5,965,875), the depths of photodiodes for detecting light, from the surface of the image pickup element are different from one another at respective three color outputs of R, G and B of the image pickup element.
The color image sensor of this type performs color separation by utilizing the differences of spectral sensitivity characteristics depending on the depths from the surface of the sensor. That is, a photodiode for outputting a B output, a photodiode for outputting a G output, and a photodiode for outputting an R output are laminated in the depth direction of the sensor to divide electric charges generated by incident light, in accordance with the entered depths. Then, the electric current of each photodiode is detected, and thereby the color image sensor obtains color outputs.
However, in the image pickup element of the configuration like this, there is a case where the output levels of the three photodiodes laminated in the depth direction are different from one another according to photographing conditions such as a change of the F number of a photographic lens.
That is, the F number proportionality of output levels of an image pickup element is closely related to the depths of photodiodes from the surface of the image pickup element. In case of a multi-layer photodiode type color image sensor, there is a case where the F number proportionality differs at every color output. In such a case, the output level ratios among respective colors differ between the case where an iris is diaphragmed and the case where the iris is opened. Consequently, a problem in which hues of an image differs in such cases is produced.
On the other hand, the spectral sensitivity characteristics of respective color outputs of an image pickup element in a digital camera configured as described above are dispersed a little owing to the differences of the manufacturing times of the image pickup elements and individual differences.
For obliterating the individual differences of the image pickup elements and for obtaining good color reproducibility, a color correction is mainly made by calibration in a camera manufacturing process in prior art. That is, adjustment values obtained by the execution of a set of a series of calibration are stored in a digital camera in advance, and corrections of images are performed by means of the stored adjustment values.
However, in the case where an image pickup element has a structure as shown in FIGS. 13A and 13B, respective color output levels of the image pickup element change owing to the dispersion of temperature characteristics of photodiodes themselves and the dispersion of the characteristics of output circuits when external factors such as temperature changes and aging are applied to the image pickup element. Then, even if the light having the same wavelength enters into the image pickup element under the same conditions, there is a case where the ratios among respective color output levels of R, G and B change. In such a case, a problem in which the hues of images output finally always change and stable image qualities cannot obtained is produced.